Confederate Memorials vs. Lynching Locations¶

In our country's current political climate, there has been a major push to remove many of the Confederate War Memorials in our country as they symbolize the systemic racism which has been perpetrated since the Civil War. These monuments have obviously emboldened racists across the country to commit heinous acts for their cause, but do these monuments act ubiquitously or does each monument do its own part in helping racism? In other words, is it because these monuments are all over the country that racist acts like lynchings occurred so frequently during the post-Civil War/Jim Crow era or does each specific monument do an equal part in facilitating these events?

In this tutorial, we will be looking into different ways to organize our data and draw conclusions from the locations of Civil War Memorials (an overwhelming amout of which are Confederate) and the locations of recorded lynchings throughout history (1877-1950). First, we want to organize all this data in a digestible form.

Part I : Data Tidying¶

This csv file contains information on every city in the United States. However, since our data for lynchings only has locations in state and county, we won't be using the city names. The useful thing this dataset does is give the latitude and longitude for every county in the US as well as the county FIPS, a unique identifier for state counties.

Next we can load in our data on the Civil War Memorials throughout the country. Here, the location is given by state and city, but again, this can easily be converted with our cities dataset.

Finally, our last data set is of recorded lynchings between 1877 and 1950. The exact definition of a lynching has been historically <a href = https://www.smithsonianmag.com/smart-news/map-shows-over-a-century-of-documented-lynchings-in-united-states-180961877/>hard to define</a>, so datasets about lynchings can be inconsistent. This file, however, contains nearly 4000 documented lynchings in the area of our interest, so we have deemed it legitimate for this tutorial.

Most of the data in this set will not be used however. Basically we just want to know which counties the lynchings took place, as indicated in the LYNCHCOUNTY and LYNCHFIPS columns.

Now we can start combining and tidying up these datasets to make them easier to use. We can merge the Civil War Memorials dataset with our cities data set so that we now have the county FIPS code for every memorial. Merging the two on state_name and city give us this mess of a dataframe. This will be greatly reduced within the next few steps.

Now we clean up the cities dataset to essentially turn it into a counties data set. We do this by removing the city name columns as well as any other columns we won't be using. Then we drop the duplicate county names and we are left with a much cleaner table.

Now we go back to our two other datasets. By using the Dataframe.value_counts() method we are able to get the number of Civil War Memorials per county FIPS and the number of lynching per county FIPS. This returns a very simple table with the counts per county code, but it is not very readable at this stage. We can begin to see a possible relationship between lynching_count and CWM_count, but the county_fips does not communicate our data very clearly.

To get our final dataframe, we do one last merge on the county_fips column. We now have a set of counties across the united states, their coordinates, the number of lynchings and the number of Civil War Memorials in each. We are finally ready to begin regression and visualization.

Part II: Regression¶

For this next part, we will want to start graphing our data. The matplotlib pyplot module is excellent for simple, yet understandable graphs. We also will want to use the numpy library in order to make easy regressions. Lastly, we will be using the scipy library to get more concrete data on the regressions we make.

Before we can graph our data, we need to extract the lynching counts and the civil war memorial counts into lists we can easily use later. Once we have those two pieces, we can use the numpy polyfit function to find the regression line which we will prepare for our graph using the poly1d function as the polyfit function simply outputs and equation for the regression.

Now that we've prepared our data and have our regression, we are ready to plot. For this, we will use the pyplot plot function which takes two lists (our x and y lists) as well as settings on how we want our data to be represented. For this, we make our points as yellow dots defined by 'yo' with y standing for yellow and o standing for dots. For our regression, we use a dotted line as defined by — and color it black as defined by k. For other formatting and argument manipulation informations check the plyplot plot function page.

Just from looking at the graph, we cannot see a real linear relationship between lynchings and confederate war memorials, but just for the sake of curiosity, we can use the scipy library's stat module and linregress function to find the exact slope, intercept, r-value, and p-value.

Based on this more concrete data, we can see that there is a slight positive slop and based on the fact that the p-value is below 0.1, this likely shows some kind of weak positive correlation between lynchings and confederate war memorials. It is possible that due to most of the counties having very few lynchings and confederate war memorials, the correlation is weaker and we might be able to see a stronger correlation from using only the counties with the most lynchings. In order to do this, we will graph the head of our sorted data frame (a sample of 30 was chosen as we felt it was not too small that it it only true outliers, but not so big that it incorporates many of the low lynching counties included in the full dataset). We also can run the scipy linregress function to see how the regression is affected.

Based on the result of the graph and the linear regression, we can see that the slope of the regression for the high lynching counties is much higher, but our uncertainty is higher as well as you can see from the p-value being much higher. Due to these limitations, it might be better to try to fit the full data for a non-linear regression. In order to do that, we simply have to change the last argument in the polyfit function from 1 to 2. This make our regression quadratic.

From this regression, we see a much more positive relationship that seems to fit that data better. Due to our large dataset (~1000 indices), we should be able to fit lines with higher orders to it without much issue. We can do this by increasing our last argument in the polyfit function more as we will do below.

As you can see from the graphs, we are able to match the data better and better by using a higher order fit with our 5th order regression fitting the data fairly well. This also shows the dangers of overfitting as even Jupyter is warning us that using 1000 points for a 100 order regression is likely to be overfit and poorly conditioned.

Part III: Visualization¶

For this section, we are going to walk through a few simple visualizations for our location based data. To do that, we will be using the library folium to plot our points on a map. The first one will be very simple: we can plot the location of every lynching compared to the location of every Civil War Monument. Arbitrarily, lynchings will be plotted as blue points and Civil War Monuments will be in red.

As we can se, there is a slight indication that lynchings occured nearby the Civil War Memorials, but the map gives little indication on the strength of this relationship. To learn more, we can create larger circles for counties with more of either count. First, we can see the distribution of the memorials across the United States

Now that we have an idea of where the most memorials are located, we can plot the lynchings and see if there is a visible relationship.

Now our visualization tells a deeper story. There is evidence of larger blue circles nearby red circles, but there are some inconsistencies. For instance, The largest red circle by Washington D.C. has very few lynching dots nearby. Conversly, we see large blue circles near Louisiana, but there is not always a red dot nearby to "explain" it as we seeked to discover.

Conclusion¶

From our data, there does not seem to be a clear geographical correlation between lynchings and Civil War Memorials, however it is possible that stronger correlations can be drawn by using different definitions of lynchings. It may also be possible to find better correlations if there was more geographical information about where lynchings took place. Due to the nature of these lynchings, finer geographical data about each lynching is ephemeral in comparison to the same data for the Confederate War Memorials. For those interested in this topic, we urge you to look for more specific datasets to use in correlations between lynchings and Confederate War Memorials. We also urge you to look into the Peace and Equality Memorial which is opening in 2018 in Alabama as it uses some of the same data we used to create a physical work of art in order to commemorate those who were killed in the name of Civil Rights in America.